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A variety of treatments have been historically used to alleviate phantom limb pain 
(PLP) with varying efficacy. Recently, virtual reality (VR) has been employed as a more 
sophisticated mirror therapy. Despite the advantages of VR over a conventional mirror, 
this approach has retained the use of the contralateral limb and is therefore restricted 
to unilateral amputees. Moreover, this strategy disregards the actual effort made by the 
patient to produce phantom motions. In this work, we investigate a treatment in which 
the virtual limb responds directly to myoelectric activity at the stump, while the illusion of 
a restored limb is enhanced through augmented reality (AR). Further, phantom motions 
are facilitated and encouraged through gaming. The proposed set of technologies was 
administered to a chronic PLP patient who has shown resistance to a variety of treatments 
(including mirror therapy) for 48 years. Individual and simultaneous phantom movements 
were predicted using myoelectric pattern recognition and were then used as input for VR 
and AR environments, as well as for a racing game. The sustained level of pain reported 
by the patient was gradually reduced to complete pain-free periods. The phantom posture 
initially reported as a strongly closed fist was gradually relaxed, interestingly resembling 
the neutral posture displayed by the virtual limb. The patient acquired the ability to freely 
move his phantom limb, and a telescopic effect was observed where the position of the 
phantom hand was restored to the anatomically correct distance. More importantly, the 
effect of the interventions was positively and noticeably perceived by the patient and 
his relatives. Despite the limitation of a single case study, the successful results of the 
proposed system in a patient for whom other medical and non-medical treatments have 
been ineffective justifies and motivates further investigation in a wider study. 
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BACKGROUND 

Phantom limb pain (PLP) is a common and deteriorating con- 
dition suffered by ~70% of amputees (Dijkstra et al., 2002), 
and regardless the cause of amputation (Clark et al., 2013). In 
recent years, virtual reality (VR) has been used to treat PLP 
as a more technologically sophisticated version of the well- 
known "mirror" therapy introduced in 1996 (Ramachandra and 
Rogers-Ramachandra, 1996). VR has clear advantages over the 
physical constraints imposed by the conventional mirror box, as 
it allows a wider range of motion and rehabilitation exercises. In 
addition, VR allows interactive games that challenge patients with 
varying levels of difficulty, while keeping them entertained and 
motivated (Sveistrup, 2004). Contemporary reviews of the use of 
VR in neuromuscular rehabilitation are given in Sveistrup (2004), 
and Holden (2005). 

To date, VR mirror therapy has relied on patients commanding 
the same motor execution in both limbs. A virtual representation 
of the missing limb is then created to match the motions of the 



contralateral limb, thus delivering visual feedback (Murray et al., 
2006a,b; Mercier and Sirigu, 2009; Bach et al., 2010). Since the 
sound limb is required, this approach is only suitable for unilat- 
eral amputees. The patients have no direct volitional control of 
their phantom limb virtual representation. Instead, they simul- 
taneously execute the same motions in both limbs. In this setup, 
the real effort and commitment of the patient to produce phan- 
tom limb motions is not part of the intervention, i.e., the mirror 
limb will move as long as the sound limb does, and regardless 
of the intention of the phantom limb. Additionally, it has been 
suggested that the variable efficacy of this therapy across sub- 
jects is mainly due to the difference in individual susceptibility to 
the visual feedback, rather than the physiological condition itself 
(Mercier and Sirigu, 2009). We hypothesize that the higher degree 
of realism provided by augmented reality (AR), together with 
direct volitional control through the prediction of motion intent 
using myoelectric signals at the stump, could improve the effi- 
cacy of this therapy. Furthermore, the addition of game control 
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by phantom limb motions should help to engage the patient in 
executing these movements and, since only the amputated limb is 
involved, it is also suitable for bilateral amputees. 

VR-based treatment in which the virtual limb is controlled 
by the affected side has been previously explored with motion 
tracking technology (Cole et al., 2009), which inherently, and 
considerably, restricts the amount of predictable motions. Here 
we show that myoelectric pattern recognition allows for the accu- 
rate prediction of hand, wrist, and elbow motions as intended in 
an intact limb. 

The utilization of the stump musculature to control conven- 
tional myoelectric prostheses has been long thought to reduced 
PLP (Lotze et al., 1999), despite that most commonly, the con- 
trolling muscle contractions are not originally related to the end 
actuation (i.e., in a trans-humeral amputee, an electrode over 
the biceps muscles controls the closing of a prosthetic hand). 
However, even if the musculature for physiologically appropriate 
actuation is no longer present, it has been shown that amputees 
are able to distinguish between imagining a phantom movement, 
and actually executing it. This suggests that the ability to naturally 
execute a movement is maintained after amputation, but more 
importantly, the effect on neuroplasticity and inter-hemispheric 
communication is different when practicing motor execution 
and motor imaginary (Raffin et al., 2012a,b). Experiments with 
implanted neural interfaces, which rely on the physiology of 
motor execution, have been shown to reduce PLP (Di Pino et al., 
2012). This supports the use of direct volitional control through 
myoelectric signals at the stump, with the advantage that the sys- 
tem presented here is non-invasive, and allows the equivalent to 
a physiologically appropriate control (i.e., muscle synergies gen- 
erated with the intention of closing the missing hand, results in 
closing of the virtual hand). 

It has been suggested that incongruencies in the visual stim- 
ulus and sensory perception produce varying results in terms of 
pain relief, in some cases increasing it (Desmond et al., 2006). 
This problem is avoided in our proposed myoelectrically con- 
trolled AR environment (MCARE), where a conventional web- 
cam captures the whole environment around the patient and 
integrates it in the rehabilitation task. To the best of our knowl- 
edge, this is the first time that AR, gaming, and the prediction 
of motion intent using myoelectric pattern recognition have been 
used together as a treatment for PLP. Comprehensive reviews of 
PLP are given in Nikolajsen and Jensen (2001), and Flor et al. 
(2006). In this work, the results of using MCARE in a chronic, 
treatment-resistant PLP patient are reported. 

A chronic PLP patient for whom other treatments have proven 
ineffective was recruited to this study. The patient (male, 72 years 
old) lost his arm just below the elbow joint in 1965 due to a 
traumatic injury. He has experienced PLP since the amputation 
and reported a strongly closed fist as the permanent posture 
of his phantom hand. The PLP has continued over the years, 
despite conventional mirror therapy, different drug-based treat- 
ments, acupuncture, and self-suggested hypnosis. The patient has 
reported living with constant burning pain of an intensity of 3 on 
a scale from 0 to 10 (SF-MGPQ; Melzack, 1987), with episodes 
that escalated up to the maximum intensity approximately every 
hour for a few minutes, reported as excruciating pain. In addition, 



the patient was normally woken at night due to intense episodes 
of pain. 

METHODS 
PAIN TRACKING 

Pain perception was monitored after every session using the 
short-form McGill pain questionnaire (SF-MGPQ) (Melzack, 
1987) translated into Swedish (Burckhardt and Bjelle, 1994). The 
questionnaire was administered by a facilitator, with the excep- 
tion of pain intensity where the patient noted the rating directly 
on the visual analogue scale. A percentage of total time at each 
level of pain was also reported. Additionally, the patient was free 
to self-report any comments on the system and the treatment. 

CONTROL SOURCE 

The prediction of motion intent was made using BioPatRec, an 
open source platform initially developed for advanced prosthetic 
control strategies based on pattern recognition algorithms (Ortiz- 
Catalan et al., 2013). The myoelectric activity at the patient's 
stump was utilized as the sole input to determine the intended 
phantom limb motions. Once the aimed motion is known, this 
can be used to command a variety of virtual environments and 
robotic devices. A custom-made AR environment was developed 
for this study to interface with BioPatRec and allow the patient to 
visualize himself (in real-time) with a virtual arm superimposed 
on his stump. The AR environment uses a conventional web- 
cam which inputs a video feed that is analyzed to track a fiducial 
marker, thus allowing the virtual arm to remain in the anatom- 
ically correct position while the patient moves (see video in 
Additional File 1). The fiducial marker can be printed with a con- 
ventional printer. The virtual arm is superimposed on the marker 
and changes scale and rotation based on the tracking of the 
marker. These parameters can be also adjusted in real-time with 
the keyboard in order to improve the fitting of the virtual arm. 

MYOELECTRIC RECORDINGS 

Eight bipolar electrodes (self-adhesive Ag/AgCl, 0=1 cm, 
and ~2 cm inter-electrode distance) and the marker were placed 
around the stump, as shown in Figure 1 . The location of the 
electrodes was defined by asking the patient to perform differ- 
ent movements and palpation of the corresponding muscular 
activity. We have empirically found that this procedure, rather 
than pre-defined selective placement, allows dealing with the dif- 
ficulties of a commonly altered anatomy at the most distal part 
of the stump. The movements requested were hand open/close, 
wrist pro/supination, wrist flexion/extension and elbow flex- 
ion/extension. 

The amplifiers used were developed in-house (MyoAmpF2F4- 
VGI8) with embedded active filtering: 4th order high-pass filter 
at 20 Hz; 2nd order low-pass filter at 400 Hz; and, Notch filter 
at 50 Hz. The signals were amplified with a gain of 2000 and 
digitalized at 2 kHz and 16 bits. 

The protocol for myoelectric signals acquisition and process- 
ing is described in Ortiz-Catalan et al. (2013). The classifiers used 
were Linear Discriminant Analysis in a One-Vs-One topology 
(LDA-OVO), and Multi-layer Perceptron in a dedicated topol- 
ogy per degree of freedom (MLP-AAM), for individual and 
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FIGURE 1 | Setup for the myoelectrically controlled augmented reality 
environment (MCARE). (A) Surface electrodes and a fiduciary marker 
placed at the stump. (B) Environment captured by the webcam and 
displayed on a computer screen, with the addition of the virtual limb 
superimposed on the fiduciary marker. (C) Patient playing a racing game in 
which he drives the car by phantom motions (Trackmania Nations Forever, 
free version). (D) Patient using the Target Achievement Control (TAC) test 
as a rehabilitation tool. 



simultaneous movements, respectively. These classifiers have been 
shown to be successful at both tasks in real-time studies and are 
further described in Ortiz-Catalan et al. (in press). 

INTERVENTION 

Once the electrodes and marker were in place, and the quality 
of the EMG signals was verified by short real-time myoelectric 
recordings, the subject was asked to perform the eight movements 
while being guided on the length and timing of the contractions 
by a virtual limb. The instructions given to the patient were to 
perform the motions "as if he still had the missing limb," thus 
aiming for physiologically appropriate myoelectric activity to be 
used for control. The LDA-OVO was trained with this informa- 
tion and the patient had a 10-min session in the AR environment 
in which the facilitator prompted the patient to perform the 
recorded movements one by one in random order (Figures 1A,B). 

After the AR environment session, new EMG recordings were 
made for simultaneous movements using wrist pro/supination 
and elbow flexion/extension. This information was used to 
train the MLP-AAM, which real-time predictions were used 
to play a racing game (Trackmania Nations Forever, free ver- 
sion). The game was controlled by using wrist pro/supination 
to turn left/right, while elbow flexion/extension controlled 
the car acceleration/deceleration. After a gaming session of 
~10min, the same procedure was repeated for hand open/close 
and wrist flexion/extension, always in combination with elbow 



flexion/extension (Figure 1C). These combinations of motions 
were also used in the Target Achievement Control (TAC) test 
initially introduced by Simon et al. (2011b), with modifications 
described in Ortiz-Catalan et al. (in press). The artificial limb 
speed was two degrees/prediction (new predictions every 50 ms) 
and the target posture was displaced in 1 and 2 degrees of freedom 
(DoF). The velocity- ramp algorithm was used to facilitate con- 
trollability (Simon et al, 2011a). In this work, the TAC test was 
used for rehabilitation and training, rather than as an evaluation 
tool (Figure ID). 

Once the TAC tasks were completed, a new set of movements 
was recorded using all eight movements to conduct a "Motion 
Test" (Kuiken et al., 2009), as implemented in BioPatRec (Ortiz- 
Catalan et al, 2013). Similarly to the TAC test, the Motion Test 
was aimed as a rehabilitation tool. Questionnaires were admin- 
istered by the facilitator at the end of the Motion Tests which 
concluded the session. 

This protocol was applied once a week starting in March 2013 
and this work includes the results up to week 18. In the last 5 
weeks, two sessions a week were held, while in week eight no 
session was conducted because the patient was unavailable for 
reasons unrelated to treatment. A video showing examples of the 
interventions is available as Additional File 1. 

RESULTS 

An increment in pain was reported by the patient after the first 
session, however, the pain decreased slightly below the origi- 
nal level in the second session, after which a slow yet consistent 
improvement was seen in the sustained level of pain. Figure 2 
illustrates the progress in pain reduction. After 4 weeks, the 
patient reported starting to experience episodes of lower pain 
intensity. After 10 weeks, episodes of almost absent pain started 
occurring and this then developed into completely pain- free peri- 
ods a couple of session later. This was reported by the patient as 
the most dramatic effect: "These pain-free periods are something 
almost new to me and it is an extremely pleasant sensation." In 
addition, pain-free periods of 15-60 min were reported immedi- 
ately after the rehabilitation sessions. 

As the patient is very active in agricultural activities, despite 
his disabilities, he performs physical tasks that involve the use of 
his prosthesis. These activities often induced sessions of pain dur- 
ing the following days. Each week, the patient reported that the 
periods of pain that normally came in the days following the activ- 
ities had been dramatically reduced and that he was able to work 
harder without being afflicted by PLR 

Surprisingly, the patient was capable of sequential control of 
three DoF from the first session, which evolved to four DoF and 
simultaneous control after four sessions. The patient reports that 
he is now able to control the motion of his phantom limb at will in 
the trained DoF. This is even possible in the absence of the visual 
feedback provided by the system, as is the case when he drives. 
More importantly, he reports being able to control (stop) the pain 
episodes considerably more effectively than before the interven- 
tions. Furthermore, he no longer wakes up at night due to PLP. 
The patient's life partner reports that it is her belief that "My hus- 
band can live 10 years more than I expected, as pain now plays a less 
important role in his life and those close to him can see it." 
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We have previously observed that patients using BioPatRec 
reported a telescopic effect on the position of the phantom limb. 
The patient initially reported the perception of his phantom hand 
at the stump height, which over the course of the treatment 
extended to the anatomically unaltered position. Interestingly, 
when he rests his arm over a table producing sensory feedback, 
the perceived position of the hand moves back to the end of 
the stump. However, as soon as he starts producing phantom 
motions, the perceived position is once again restored to the 
unaltered anatomy. This is a phenomenon that is now perma- 
nently present and it indicates the complexity of self-perception 
and how it can be altered by sensory feedback and motor execu- 
tion. It is worth noting that the presence of a phantom limb map 
on the stump is weak, mixed, and fairly difficult for the patient 
to identify, thus providing limited sensory information from the 
phantom hand. 
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FIGURE 2 | Evolution of pain intensity over time. (A) The distribution of 
pain intensity over time shows that at the beginning of the treatment, the 
patient had a sustained level of pain (~30%) during more than half of the 
time, and periods with higher levels of pain the rest of the time. Over the 
course of the treatment, a reduction of time at higher pain intensity levels 
was reported, as well as the appearance of periods of lower or absent pain. 
(B) The sustained level of pain was also the lowest pain perceived by the 
patient, and it gradually decreased to around 10% over the course of the 
interventions. Episodes of reduced pain started occurring after 4 weeks of 
treatment and gradually became pain-free periods. In week 11, a problem 
with his socket prosthesis caused him to use an old, tighter socket that had 
previously been shown to induce pain. 



The initial state of the phantom hand was described by the 
patient as a permanently, strongly closed fist and this has been 
the case for the last 48 years. After six sessions, this state evolved 
to a mid-open hand position, which coincides with the neutral 
(relaxed) position shown by the virtual hand. This is now the 
permanent perception of phantom hand posture and it is greatly 
appreciated by the patient (patient self- report). We do not have 
enough data to argue that the constant visualization of such posi- 
tion as the normal virtual state has influenced its perception as the 
default phantom limb state, or whether this is instead the result 
of the patient's skill at moving the phantom limb. In any case, the 
relaxation of such a stressed position occurred at the same time as 
the appearance of reduced pain periods and it could therefore be 
attributed as one of the causes of reduced PLP. 

As expected, the ability of the patient to control the different 
motions improved over the course of the treatment. It is worth 
mentioning that no muscles directly responsible for the more dis- 
tal movements were available due to the level of amputation (e.g., 
hand open and close). However, the patient was capable of vol- 
untarily controlling the virtual limb to produce those motions. 
We hypothesize that the patterns of myoelectric activity produced 
by muscle synergies are distinctive enough to allow the classifiers 
to differentiate directly related movements from those occurring 
more distally at the hand. Figure 3 illustrates the learning curve 
through the improvement of the classification accuracy of nine 
classes (eight movements plus "no movement"). In this case, the 
classification accuracy indicates how well motions can be dis- 
cerned from each other using information from the recorded 
sessions (offline), whereas the real-time performance once the 
patient has acquired experience with the system is shown in 
Table 1. Despite the relative low level of offline accuracy at the 
beginning of the treatment, the interventions were still possible 
because only a few movements were discriminated together for 
each rehabilitation task, thereby making the differentiation easier 
for the classifiers, i.e., only two DoF (four movements) were used 
for game control and the TAC test. 

The motion test results after week 15 show performance com- 
parable to that of 17 able-bodied subjects previously evaluated 
(Ortiz-Catalan et al., 2013), where the signal processing, features 
and motion test settings were the same. Although these results 
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FIGURE 3 | Offline accuracy. The offline discrimination accuracy over time 
is presented in box plots where the central mark represents the median 
value; the edges of the box are the 25th and 75th percentiles; the whiskers 
give the range of data values; "*" represent average values. 
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Table 1 | Motion test results. 





PLP patient 


BioPatRec study 


# Movements 


8 


10 


# Electrodes 


8 


4 


Selection time (s) 


0.56 (±0.14) 


0.62 (±0.24) 


Completion time (s) 


1.71 (±0.15) 


1.86 (±0.31) 


Completion rate (%) 


98.0 (±2) 


87.4 (±11) 


Real-time accuracy (%) 


75 (±4.2) 


67.1 (±10) 



cannot be compared directly, they serve as an indication of the 
ability of the patient to produce distinctive motions in real-time. 
It is worthy of notice that the number of electrodes has been 
reduced to four since this report without a noticeable effect in 
classification performance. 

DISCUSSION 

A complete understanding of the root causes and underlying 
mechanisms of PLP has evaded the scientific community for 
decades (Nikolajsen and Jensen, 2001; Flor et al., 2006). This 
understanding will undoubtedly enable the creation of more 
effective treatments for the condition. In this context, the sys- 
tem proposed here provides empirical information on the effect 
of reactivating brain areas related to motor execution, enabling 
visual feedback that "tricks" the brain into believing that there 
is a limb responding to motor commands, and exercising the 
stump musculature, which is normally neglected. Mirror therapy 
is based on the assumption that visual feedback can potentially 
correct tactile deafferentation (to some degree) due to brain plas- 
ticity. Evidence has been reported on the correlation between 
cortical reorganization and PLP (Flor et al., 1995), which was fur- 
ther investigated to argue that extensive myoelectric prosthetic 
use prevents it, and thus reduces PLP (Lotze et al., 1999). The 
patient reports wearing his body-powered prosthesis all the time 
he is awake and he has done so for decades. In this case, although 
limited visual feedback is provided by the prosthesis, the pros- 
thesis does not respond to physiologically appropriate commands 
and the motion of the missing limb is thus neglected. This might 
explain why, although the patient has used his prosthesis exten- 
sively, the PLP has remained. This underlines the importance of 
a congruent relationship between feedback and motor execution, 
as well as the intention to perform motion execution itself. All this 
is synergistically provided by our system. 

In this case study, we present a system that can be used for PLP 
treatment and has had relative success in a patient with chronic 
PLP who had unsuccessfully explored several other treatments. 
Despite the fact that the pain has not disappeared completely 
at the time of this report, its reduction and temporal absence 
have considerably improved the patient's condition (patient self- 
report). It still remains to be seen whether the pain disappears 
completely after the long-term use of the system. The ideal med- 
ical treatment would be to administer it for a defined period of 
time and permanently cure the condition. We have intentionally 
avoided terminating the sessions to evaluate the long-term effect, 
as we feel this would be unethical, given the satisfaction reported 
by the patient after 48 years of chronic pain. As an alternative, the 



patient has been provided with a stand-alone system to be used at 
home and he has been instructed to use it at his own discretion. 
Follow-ups will be conducted every 2 months for a year and every 
year after that for 5 years. 

The combination of myoelectric control of a virtual limb using 
physiologically appropriate signals, the enhanced illusion given 
by AR, and the entertainment provided by gaming has enabled 
the patient to develop the skill to control the motion of his phan- 
tom limb at will, even outside the lab. It is not clear whether this 
skill alone is enough to reduce PLP, because (1) this was acquired 
through visual feedback forcing the brain into the illusion that the 
limb is present, thus facilitating phantom limb motions (Brodie 
et al, 2003); and (2) the intervention inevitably results in motion 
intent and a workout of muscles at the stump which are nor- 
mally neglected. It has been argued that the second factor alone 
is a cause of PLP relief (Sherman, 1980). The independent con- 
tribution of these two factors could be difficult to isolate in 
the proposed system. Motor intention alone has been shown 
to similarly reduce PLP when comparing mirror therapy with 
and without visual feedback (Brodie et al, 2007). When visual 
feedback was used, however, the capabilities of phantom limb 
motion increased. On the other hand, VR interventions where 
the controlling side is the amputated has shown signs of PLP 
relief, despite that the musculature at the stump was not directly 
involved (Cole et al, 2009). Treatment-wise, the combination pre- 
sented here including all the latter was successful in a particular 
but complicated case, and it requires further investigation in a 
wider clinical study. 

The possibility of decoding distal movements using muscles 
synergies was initially explored decades ago (Wirta et al., 1978). 
In 1982, Saridis and Gootee used pattern recognition to decode 
wrist pro/supination from biceps and triceps muscles (Saridis and 
Gootee, 1982), however, they reported that they were not able 
to decode hand open/close; possibly due to the limited number 
of electrodes used (2 bipolars). In our experience, patients can 
quickly learn to control a few distal motions and the results pre- 
sented here suggest that they are able to develop that skill further 
to several motions. It is worth noting that one limitation of this 
non-invasive approach is that a certain degree of musculature 
is required, i.e., shoulder disarticulations would hardly be treat- 
able unless they were recipients of targeted muscle reinnvervation 
(Kuiken et al, 2004, 2009). On the other hand, the proposed 
treatment can be used seamlessly in any patient requiring neu- 
romuscular rehabilitation, in cases such as stroke and incomplete 
spinal cord injuries (Lee et al, 2011; Liu and Zhou, 2013), again, 
given the availability of myoelectric signals. 

VR treatments are commonly justified and encouraged by the 
assumption that sensory stimulation boosts neuromuscular reha- 
bilitation. At the current stage, the system employs only visual 
feedback stimuli, mostly due to the technical difficulties involved 
in providing proper somatosensory stimulation. In our experi- 
ence, patients invariably prefer a virtual limb to any other visual 
feedback and we are therefore presently developing rehabilita- 
tion games based on AR that are specifically designed to exercise 
selected motions in a controlled manner. 

The proposed system incorporates different advantages of 
computational rehabilitation systems, such as progress tracking, 
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adjustable task difficulty, engaging rehabilitation tasks, and porta- 
bility. Furthermore, the VR environment and all the source code 
necessary for motion prediction using sEMG (including game 
control) are freely available and open source in BioPatRec (Ortiz- 
Catalan et al., 2013), which aims to enable researchers worldwide 
to use this technology. 

CONCLUSIONS 

PLP has historically being a difficult condition to treat and it 
affects the majority of amputees. In this work, we introduce a 
non-invasive technological proposal that combines the prediction 
of motion intent through the decoding of myoelectric signals, 
virtual and augmented reality, and gaming. As opposed to con- 
ventional mirror therapy, this system allows full range of motion 
and direct volitional control of the virtual limb, and it is appli- 
cable for bilateral amputees, in addition to having the known 
motivational benefits of gaming and progress tracking by com- 
puterized systems. This system is presented with a case study 
of a chronic PLP patient with known resistance to conventional 
PLP treatments. Having shown that the system has considerably 
increased the quality of life of a single patient, where other previ- 
ous conventional treatments had proved unsuccessful, we believe 
that it offers sufficient justification to further explore its efficacy 
on a wider PLP population. 
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